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ABSTRACT 


Interaction is crucial in classrooms because increased interaction is linked to 
increased learning. Past studies report that students learn by a myriad of methods, and 
that it is up to the instructor to promote as many means as possible to transport the 
material to the students. One way in which instructors are providing information to their 
students is through a classroom response system (CRS), an electric transponder the size 
of a remote control. The CRS allows users to respond and interact with the push of a 


button. 


This study looked at educational institutions using CRS, in order to identify the 
distinctive characteristics that are analyzed to value its effectiveness in a classroom 
environment. The information collected was examined to gain an understanding of the 
various uses of CRS to determine if they would be a beneficial addition to resident NPS 


curriculums. 


Also, this study employed a posttest-only independent group quasi-experimental 
design to test the effects of clickers in the classroom. Specifically, clicker use was studied 
to determine what impact, if any, their use would have on student interaction in the 
classroom, student engagement, student motivation, perceived teacher immediacy, course 
liking, and students’ overall evaluation of the clickers. The findings and implications of 


this study are discussed. 
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I. BACKGROUND AND LITERATURE REVIEW 


A. CHAPTER OVERVIEW 


The purpose of the case study is to look inside educational institutions that are 
using classroom response systems (CRS) and identify the distinctive characteristics that 
are analyzed to value its effectiveness in a classroom environment. The goal is to gain an 
understanding of the various uses of CRS to determine if they are worth implementing 
into resident NPS curriculums. This chapter answers the question: what are CRS? It then 
describes their general characteristics, examines the motivations for their use, and 


summarizes the more popular models available on the open market. 


This chapter first defines CRS by breaking down and describing the individual 
components that make up the CRS. CRS have also been referred to as personal response 
systems, classroom communication systems, group response systems, electronic voting 
systems, audience paced feedback systems, classroom network systems, and audience 
response systems. Second, this chapter explores why CRS are used in institutions by 
delving into the historical developments and outlining the different environments in 
which they are used. Third, this chapter references case studies from institutions that used 
CRS in large classroom environments (e.g., West Virginia University, Wayne State 
University, Surrey University, Eastern Washington University, and Ohio State 
University), in order to discuss methods in which they statistically proved the 
effectiveness, typical uses, and characteristics of CRS. Finally, this chapter compares and 


analyzes five CRS from the most popular and commonly used CRS manufacturers. 


In the past couple of years, large advances in CRS have been made in both 
technology and popularity. The Naval Postgraduate School (NPS) mission statement 
states that NPS “provides relevant and unique advanced education...” (Registrar, 2007). 
Operating on the cutting edge of technology, NPS has recently implemented the use of 
CRS into their Executive Management curriculum to provide a more unique and 


advanced education. Always striving for technological and educational improvements, 
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the NPS Graduate School of Business and Public Policy is experimenting with the use of 
the Renaissance Classroom Response System. The results from analyzing case studies are 
used to formulate the methodologies for CRS research study (Chapter II), which is used 
to test and evaluate the level of student interaction with CRS in resident NPS 


curriculums. 


B. DESCRIPTION OF CLASSROOM RESPONSE STYSTEMS (CRS) 


1. CRS Overview 


Simply stated, the handheld devices in classroom response systems (CRS)— 
commonly known as “clickers” or “key-pads” in the United States and “handsets” or 
“zappers” in the United Kingdom (Barber & Njus, 2007)—are electronic transponders 
about the size of a television remote control. Each clicker allows the user to respond by 
the push of a button. The clicker is only one of three components that make up the CRS. 
The second component is the receiver, which is typically connected to the professor’s 
computer. The receiver collects responses from the clickers. Finally, the third component 
of the CRS is the associated software. Associated software packages allow professors to 
create questions, send notes and receive clicker responses so they can be displayed for 
review. A further breakdown of three CRS components should be helpful in 


understanding why clickers are used. 


2 Description of Clickers 


Clicker handsets come in a variety of shapes, colors and sizes, and have various 
functionality differences. In a standard classroom environment, each student will have 
one clicker. Due to equipment limitations, students may have to share. Depending on the 
manufacturer and model, clickers vary in size. They can be as small as a credit card or as 


large as an old style television remote control. 


Early models of clickers had a single response button (Poulis, Masen, Robens, & 


Gilbert, 1998), while modern clickers usually have a 10-digit numeric keypad and often 


some accessory buttons including a power switch, a send button, or function keys that 
permit text entry (Barber & Njus, 2007). Early models only allowed students to respond 
to yes/no, true/false or A/B types of questions. These older models also had a wire that 
connected the clickers to the receiver. The advantages to the simplicity of earlier models 
are that “they required little effort to understand the system, there was no possibility of 
the wrong button being pressed, and they caused minimal disruption and distraction when 
used” (Poulis et al., 1998). The primary disadvantages were that, if students had to 
answer more than one question, response options had to be manually entered 
sequentially, preventing the user from skipping a question and returning to it later. 
Additionally, earlier models only provided one-way communication, meaning students 
could send their responses but there was no indication of whether the response was 


received. 


Aside from having the luxury of being wireless, modern clickers have many more 
capabilities through their multi-button keypads. Not only can they answer the simple 
yes/no questions, modern clickers enable students to answer multiple-choice questions 
and express confidence levels in the accuracy of their answers, which is helpful in 
analyzing whether correct answers were chosen through luck or based on knowledge 


(Simpson & Oliver, 2006). 


Each clicker has an assigned ID number that allows professors to map ID 
numbers to individual students. This feature can be helpful if professors want to keep the 
identities of student responses confidential or, conversely, use them for analysis. For 
example, if professors chose to use clickers to facilitate graded quizzes, assigned ID 
numbers would allow them easily to tally and record quiz results. Finally, at the end of 
the class, depending on the situation, if professors needed to collect the clickers for 


redistribution in another class, assigned ID numbers would help with accountability. 


The most modern wireless clicker models, such as the Renaissance CRS (which 
will be used in this case study), have LCD screens that can be used to view transmitted 
text. LCD screens are useful because they allow students to read questions at their own 


speed. Professors can send notes in addition to quiz questions which can be stored in the 
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clickers’ memory and viewed on the LCD screens. Additionally, LCD screens allow 
students to return and review answers in multiple question quizzes. Having clickers with 
LCD screens frees up a media asset in the classroom. Now, professors can use the clicker 
LCD screens vice writing questions on the blackboard or posting them on a projection 


screen. 


3. Description of Receiver 


Clickers transmit data to the receiver using radio frequency (RF), infra red (IR), 
or through a wire, depending on the type of CRS. Receivers are typically connected to 
the professor’s personal computer (PC) through a USB port. They are responsible for 
capturing responses from clickers and transmitting information sent by the professor such 


as assignment questions, quiz questions, and notes. 


The word “receiver” is a misleading term for the name of the receiver component 
in a CRS. The American Heritage Dictionary of the English Language defines “receiver” 
as a device or apparatus that receives electrical signals (Berube, 2006). For early CRS 
models, receiver was an appropriate name because these early models operated using 
one-way communication. Data were transferred from the clicker and received by the 


receiver through a wire that connected the two components. 


Modern CRS use two-way communication. The receiver receives data from the 
clickers and transmits responses in return. Therefore, a more appropriate descriptive 
name for modern receivers would be transceivers. “Transceiver” is defined by the 
American Heritage Dictionary of the English Language as a transmitter and receiver 
housed together in a single unit and having some circuits in common, often for portable 
or mobile use (Berube, 2006). For the purpose of this case study, the term “receiver” will 


be used when referring to the receiver component of the CRS. 


Wireless clickers operate using one of two technologies, RF or IR. RF systems are 
rapidly becoming the current standard because they have greater range, send stronger 
signals, require only a single receiver, do not experience interference from classroom 
lights or other [R-emitting equipment and do not require a direct line of sight between the 
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student and the receiver (Caldwell, 2007). Until recently, clickers using RF signals have 
always been better suited for large audiences, but their high cost prohibited widespread 
application. However, all major CRS vendors have now introduced low-cost RF keypads, 
and many vendors have already discontinued their old IR models (Barber & Njus, 2007). 
As technological advances lower the cost of electronics, IR systems are being phased out 


and replaced by the more capable RF clickers. 


Receivers do have their limitations, depending on manufacturer make and model. 
For example, the Reply system, which uses RF, needs only one receiver for up to 650 
clickers. The PRS system, which uses IR, needs one primary receiver for the first 50 
students and a secondary receiver for every additional 35 students (Simpson & Oliver, 
2006). The added reliability, reduced cost, and increased functionality of RF systems 


have made CRS more practical to use in educational institutions across the nation. 
4. Description of Associated Software 


Each CRS comes with an associated software package. Software packages have to 
be loaded on each PC that will be used as a host for the connected CRS receiver. Due to 
modern IT restrictions, it may be necessary to contact the IT department and get 
permission to load software on school computers, and also to ensure the host’s PCs meet 
CRS software requirements. Most software systems are said to be easy to use with only 
an “intermediate” level of computer skill, thereby freeing the professor to consider 
pedagogy rather than technical operations (Brewer, 2004). User-friendly software 
shortens the training time required for professors to implement CRS into their 
classrooms, making the technology more appealing. Furthermore, easy-to-use software 


minimizes fear and stress caused by resistance to change. 


Among the variety of software packages, two characteristics emerge in the more 
commonly used CRS. First, once the receiver collects the data from the clickers, the 
software compiles and evaluates the responses, making them available for display to the 


class using some type of projection device. Responses can be displayed graphically as a 


list, table, histogram, bar-chart, or simple percentage. Secondly, software allows 


professors to create, format, and display questions. 


In addition to the two most common characteristics, software applications can be 
used by professors to perform more advanced functions, such as selecting which answers 
are correct and assigning weights to graded responses. These functions are typically 
referred to as grading tools. More complex software packages allow for real-time cross 
tabulation, making it possible to compare responses between two questions, such as pre- 
and post-test questions, or to group responses by demographic variables. These results 
can be saved and used by professors for grading or further evaluation after class (Kaleta 
& Joosten, 2007). Another example, the PRS software can record information, including 
the number of attempts made, the time taken to register the answer, and the confidence 
levels. These results can be saved and used later, employing standard software such as 


Microsoft Office to view the results (Simpson & Oliver, 2006). 


Before prematurely buying the first CRS that captures the buyer’s attention, 
conduct a little research to ensure that the associated software package that comes with 
the CRS will be able to perform the required functions. Although manufacturers typically 
have additional software packages for sale that may suit the educational institution’s 
needs, these software packages are largely dependent on the type of system purchased. 
Additional software package options may be limited. The primary drawback is that CRS 
only work with the software applications designed for each specific system. Thus, if 
some someone were to purchase a CRS that did not have the specific features required, 


the only solution would be to purchase a different CRS system. 


C. EVOLUTION OF CRS AND THEIR ENVIRONMENT 


1. Historical Overview 


The Greek philosopher Socrates was a master at pedagogy. He realized that 
people understand more by answering a question than by being told an answer 


(Abrahamson, 1999). An amazing revelation, even in this modern day and age, is that 
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society can turn back to seek wisdom from a man who lived 2400 years ago. This ancient 
wisdom is familiar to any parent today: as a father sits at the kitchen table watching his 
daughter do her homework, she looks over to him and asks for help with a simple math 
problem. The father knows that answering the question would be simple, but doing so 


would deprive the daughter of the satisfaction of figuring out the answer herself. 


The reason that the Socratic Method works in teaching is because a teacher, 
through questioning, can spotlight an area of knowledge, encourage students to think 
through the issues, establish positions, and commit to positions (Abrahamson, 1999). 
Professors asking the right questions at the appropriate moments stimulate thought 
processes. Taking stimulated thought processes and providing rapid feedback can 


reinforce this loop (Simpson & Oliver, 2006). 


The main problem with Socratic teaching is that it works well in small groups of 
five or fewer students, but in larger classroom environments the majority will be left out 
of the interaction experience (Abrahamson, 1999). Typically, professors can establish a 
one-on-one dialogue with only a few students in a large classroom during any given 
lecture, due largely to lecture time constraints. Educational institutions clearly need to 
find innovative ways of keeping dialogue channels open between the professor and 
students. This challenge is becoming more difficult as the world’s population grows, 


resulting in larger class sizes (Bartlett, 1998). 


2. CRS Environment 


In modern times, ensuring that everyone has a chance to receive an education is 
made possible by using much larger classroom environments. The method of teaching is 
quite different from years past, when the rich were the only individuals who received an 
education (Abrahamson, 1999). So, using the best teaching methods would only be 
appropriate. The tutorial system at Oxford and Cambridge in England, where two to five 
students meet regularly with their professor ("tutor") in his or her study is an excellent 
example of an active learning environment. This is clearly a wonderful environment for 


teaching and learning, but it is expensive—much more expensive than a big class or 
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lecture. This was the reason behind the CRS (Abrahamson, 1999). CRS were designed as 
a practical solution to increased classroom sizes. Although smaller classes would be 
better, CRS attempt to bridge the gap between the lack of teacher-student interactions in 


large classrooms and build a better learning environment. 


In a perfect world, searching for different methods of teaching would not be 
necessary. The passive traditional teaching approach would suffice. Professors would be 
able to give a lecture or lesson; students would listen, take notes (if needed) and, when 
quizzed, be able to recite or transcribe any part of the lecture or lesson verbatim. 
Unfortunately, this is not the case; educational experts are continuously trying to find 
new solutions to improve educational teaching methods (Schank & Jona, 1991). 
Educational research has shown that students who are actively involved in the learning 
activity will learn more than students who are passive recipients of knowledge (Kumar, 
2003). Using this research, many universities such as Wayne State are trying to bring 
active learning into the classroom using CRS (Barber & Njus, 2007). CRS is only one of 


many solutions to increase active learning in classrooms. 


Clickers are used in education to combat student attrition, increase participation, 
and change the passive, one-way communication that is common in a lecture 
environment into an active, two-way communication between professor and student. 
Combating student attrition at NPS is not an issue because NPS is not a traditional 
university. Students at NPS are mandated to go to class; therefore, using clickers for this 
capacity is not required. On the other hand, clickers can be used to increase classroom 


participation by students at NPS. 


A majority of professors at NPS use a creative approach to encourage student 
participation by incorporating a ten percent participation grade into their overall course 
grade. The participation grade is designed to motivate students to speak up during class 
by asking questions, stating their comments or opinions, telling the professor when they 
do not understand something, and essentially participate when appropriate. Numerous 
problems arise from this approach. First, classrooms of significant student size would 


suffer a reduction in the lecture time if everyone were to participate. That is, lectures are 
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not long enough for everybody to speak their mind. Second, as Caldwell (2007) observes, 
“Students in classrooms are often hesitant or unwilling to speak up because of fear of 
public mistakes or embarrassment and fear of peer disapproval.” Clickers enable students 
to voice their concerns in confidence. Unfortunately, situations exist where voicing one’s 
concern in confidence can be counterproductive to the educational environment. For 
example, if a professor chooses not to address a student’s concern, other students may be 
at a loss by not knowing the answer. Additionally, the students who choose to speak up 
often become the voice for the rest of the class. Finally, more simple solutions exist for 
combating low class participation than using CRS. For example, professors can ask the 
class to vote by raising their hands, clap, or use some other type of feedback to answer a 


question. 


These simple solutions are not without their disadvantages. When students are 
asked to raise their hands, not wanting to feel like an outsider, fellow classmates will 
often raise their hands to follow the majority vote. Occasionally, students vote this way 
because they do not know the answer. Waiting to see how the majority votes and 
following their lead is the safest approach, because the few who decide to vote against the 
majority are typically asked by professors to explain why they voted the way they did. If 


these few do not know the correct answer, peer embarrassment is sure to follow. 


To move from a passive state of learning to an active state of learning, students 
have to remain interested in what they are learning. How does a professor make lectures 
interesting and exciting to keep students’ attention? Roger Schank and Menachem Jona 
believe “experience is the best teacher” and “the best teachers are typically the best 
storytellers” (Schank & Jona, 1991). Keeping students interested requires actions such as 
the professor telling relevant stories, having class discussions, participating in group 
activities, showing movie clips or even using CRS. No matter which course of action one 
takes, the professor must sacrifice lecture time, which is not always possible. 

A great many lecturers are reluctant to accept claims on the merits of 

activating instruction, which can be found in educational theory. 


Activating students requires time, which lecturers would normally devote 
to lecturing. They often voice the concern that they will not get enough 
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material across when giving interactive lectures and that this consequently 
will negatively affect the student learning (Van Dijk, Van Den Berg, & 
Van Keulen, 2001). 


Why should professors change their teaching methods now when they have 
worked for over 100 years? Traditional teaching practices using the passive learning 
approach have been used since the early 1900s (Schank & Jona, 1991). The problem is 
that the mission of the schools of the 1900s was primarily to train good factory workers, 
not well educated citizens (Apple, 1990). Many current teaching practices arose from 
technological factors and practical constraints that no longer apply (Schank & Jona, 
1991). Clickers are an excellent example of how the tools of technology are shaping the 
educational future. With advances in technology, “students in the future will see a major 
restructuring of our social, industrial and educational institutions, and an increased 
reliance on computers for work and education” (Molnar, 1997). The difficulties in the 
road ahead lie in learning new technologies and getting professors, staff, and faculty to 


support the implementation of change. 


CRS provide practical solutions to many of the issues caused by lack of 
participation, comprehension, and passive learning environments. First, when time 
constraints are of vital importance, CRS enable professors to poll small or large student 
size classrooms using CRS questioning, tally the responses, and display the results for 
discussion. This method allows professors to break up the monotony of long boring 
lectures by adding interactive solutions in which all students can participate. Second, 
professors do not have to discuss the results immediately; they have the choice to use 
these records later for cataloging attendance, student tutorials, lesson planning, or 
educational research (Caldwell, 2007). Lastly, clickers can be used to ensure 
comprehension of difficult concepts. Professors typically break complex concepts into 
subparts. Clickers can be used to test conceptual comprehension of individual subparts to 
ensure student comprehension before continuing. The danger in this approach lies in the 
hands of the professor. By not asking the right questions, CRS technology would have 


little impact on increasing comprehension. 
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3. Additional Insight 


a. Student Motivation 


Motivation has been described as a process that includes specific directive 
and stimulating properties (Brophy, 1987). This can lead students to arousal and 
instigative behaviors, give direction and purpose to their behaviors, allow behaviors to 
persist, and lead to choices of preferred behaviors (Dweck, 1986). A general pattern of 
student motivation toward learning often takes the following sequential form: student 
energy, volition, direction, involvement, and completion (Wlodkowski, 1978). If one area 
of student motivation breaks down, the entire process may come to a complete halt 


(Christophel, 1990). 


Wlodkowski identifies five beliefs behind motivational theory that make 
the concept difficult to understand. These five beliefs reveal some of the challenges for 
measuring motivation in a classroom environment, which becomes further complicated 


when adding CRS. The five beliefs are as follows: 


First belief: When students are not willing to involve themselves in class 
activities or assignments, they are unmotivated. Although students may not be 
motivated to learn, they are usually motivated to do something. If that motivation is not 
directed toward learning, it is likely to be directed toward disruptive behaviors 
(Christophel, 1990). CRS is designed to motivate students by creating a more active 
environment, but a more active environment sometimes requires increased teacher 
control. Increased teacher control, especially in larger classrooms, may be required to 


keep unproductive discussions to a minimum. 


Second belief: Teachers motivate students. Although no one person can 
claim sole responsibility for motivating another person, teachers can make learning 
attractive and stimulating, provide opportunities and incentives, allow for development, 


and match student interests (Christophel, 1990). Motivating students is not a science. 
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Teachers must continue to find innovative ways and ideas to stimulate individual 


students. Obviously, the larger the class size, the more difficult this task becomes. 


Third belief: Since students must learn in order to survive, making them 
learn is more important than their motivation to learn. If learning is associated with 
coercion it can become a generally aversive stimulus, one that students will go out of 
their way to avoid (Christophel, 1990). Forced learning or guided learning is necessary to 
some degree. Students must be challenged, not forced or coerced to learn more. CRS is a 
tool in which teachers can challenge students to compete against one another and offer 


rewards for specific behaviors. 


Fourth belief: Threats can facilitate student motivation to learn. As 
military officers and senior DoD employees, threats are not necessary at NPS. The 
majority of the population realizes that failure at NPS will have an adverse effect on their 
existing careers; therefore, their motivation to succeed is higher than younger traditional 
college students. Using threats only stimulates students to become frightened and 
resentful of the threats and the person using them. The long term outcome is student 


avoidance of the teacher and the subject matter (Christophel, 1990). 


Fifth belief: Learning automatically improves with increased student 
motivation. Motivation is not a panacea for instruction, but it may provide a foundation 
for effective instruction (Christophel, 1990). Students do not have to be motivated to 
learn. Often, students who do not like a particular subject will learn just enough to get by 


and stay unmotivated throughout the entire process. 


Identifying the effectiveness of CRS as a motivational aid in the classroom 
is difficult. CRS provides students with feedback which can lead to educational 
reinforcement, but evaluators must comprehend Wlodkowski’s five beliefs and 
understand how student attitudes in the classroom environment can affect motivation. A 
valid measurement must be created and evaluated to test motivation in CRS classrooms. 


Based on these concepts, the following research question was proposed: 
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Can motivation be used to improve student learning with clickers? 
b. Student Engagement 


Students’ active engagement with ideas and applications supports 
learning. This view is most clearly advocated by Laurillard (Laurillard, 2002). The 
principle underlying this framework is that learning results in the process of ongoing and 
adaptive dialogue between teacher and student, supplemented by activities that provide 
an opportunity to apply ideas or practice skills (Simpson, 2006). Student engagement is 
one of the sure ways by which professors can guarantee students are actively learning. 
The problem with this approach is that only one student can participate through dialog 
with the professor at any given time in a traditional classroom environment, which 


becomes increasingly challenging with larger class sizes. 


Knowing the difficulties to achieve 100 percent student engagement in 
larger class sizes during any given lecture, teachers still encourage participation from 
each student. One solution to overcome this difficulty is to divide classes into small- 
groups. Dividing classes into small-groups will grant each student an opportunity to 
participate within their group, thereby accomplishing the goal of 100 percent 
participation. Participation in small-group discussions primes students to be more 


attentive and involved in subsequent whole-class discussions (Beatty, 2004). 


CRS can provide an additional aid to small-group discussions by giving 
each student a vote in the small-group’s final response to a question presented by the 
professor. For example, suppose a particular class was presented with the question: is the 
earth round or flat? The class would be divided into small-groups for discussions among 
group members to formulate an answer. Once small-groups have formulated their 
answers, each student would select the appropriate answer by using their clickers. 
Responses would be rapidly tallied and displayed on the overhead projector. The 
professor could quickly identify incorrect answers and address any misconceptions. By 
having students communicate their knowledge and reasoning, in small-groups and 


through class-wide discussion, CRS can help them sharpen their vocabulary, clarify their 
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thinking, discover gaps and contradictions in their understanding, and identify flaws in 
their logic (Beatty, 2004). Although a novel idea, small-group discussions are time 


consuming and not always possible. 


By their nature, CRS engage all students in classroom instruction by 
allowing all the students to respond to every question asked by the teacher. Additionally, 
students can use the CRS as opposed to raising their hand to let teachers know they do 
not understand the material being taught or have a general question. The idea behind CRS 
is not new—teachers have used interactive, instructive questioning to engage students 
since at least the time of Socrates (Caldwell, 2007). CRS allow students to become 
engaged in classroom discussion through an interactive education without the fear of 
being singled out. CRS does not prevent students from using traditional means of 
communication such as raising their hands to ask a question or voicing that they do not 
understand a particular idea. Based on these concepts, the following research question 


was proposed: 


Can student engagement be used to improve student learning with clickers? 
Cc. Teacher Immediacy 


Several studies have been conducted that analyzed the use of teacher 
immediacy behaviors (Christophel, 1990). Immediacy is the use of communication 
behaviors to enhance closeness between communicators (Mehrabian, 1969). 
Unfortunately, studies have not been conducted to research whether teacher immediacy 
can be used to analyze the effectiveness of student learning with clickers in the 


classroom. 


The applications of immediacy to educational settings introduced the idea 
that a teacher, through the use of certain cues, could reduce the perceived distance 
between instructor and learners and thereby influence certain classroom outcomes, 
especially student learning (Allen, 2006). CRS facilitate reducing perceived distance 
between instructor and learner by creating active, rather than passive, two-way 
communication channels. Using teacher immediacy as a measuring tool to evaluate the 
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effectiveness of CRS in the classroom is important because teacher immediacy represents 
a set of behaviors that an instructor can be trained to exhibit and/or increase. If a teacher's 
immediacy behaviors predict the level of student learning, then a modification of 
instructional communication behavior increases the level of learning (Allen, 2006). By 
uncovering the importance and effects of teacher immediacy in CRS classrooms, teachers 


can modify their immediacy behaviors to increase active learning in CRS classrooms. 


Teacher immediacy can further be broken down into verbal and nonverbal 
aspects. Nonverbal teacher immediacy includes behaviors such as smiles, nods, body 
posture, gestures and eye movements. Results revealed a significant relationship between 
these teacher nonverbal immediacy behaviors and students' affective learning (Witt, 
2004) The use of clickers may detract from nonverbal teacher immediacy because 
implementing clickers in the classroom adds another media device which will require 
students’ attention. This additional requirement will decrease time spent by students 
focusing on the teacher, therefore decreasing nonverbal teacher immediacy. A safe 


assumption can be made that CRS will reduce nonverbal teacher immediacy. 


Verbal immediacy received little attention in instructional communication 
research until Gorham (1988) reported a moderate correlation between verbal immediacy 
and both perceived and affective learning outcomes (Witt, 2004). Verbal immediacy 
includes but is not limited to behaviors such as addressing students by name, using humor 
in the classroom, discussing personal experiences, and asking or answering questions. For 
all of these behaviors, CRS can be used to modify the classroom environment to increase 


vocal expressiveness. 


Together, verbal and nonverbal immediacy behaviors that a teacher 
displays in communicative acts and interactions with students can be seen as rewarding. 
These rewarding behaviors may serve as reinforcement for the attentive behavior, 
feedback, and interaction from the student that increases affective, cognitive, and 
behavioral learning (Allen, 2006). Implementing CRS as an interactive tool in the 


classroom environment will increase the will of students to approach and engage in the 
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educational discussions that are critical to the learning process. Based on these concepts, 


the following research question was proposed: 


Can verbal and nonverbal immediacy be used to improve student learning with 


clickers? 


D. USES AND CHARACTERISTICS OF CLASSROOM RESPONSE 
SYSTEMS 


CRS can be used in virtually any size classroom. Due to the rationale for their 
creation, to make up for a lack of teacher-student interaction in the classroom, the 
preferred use for CRS is in classes ranging from 15 students to more than 200 students 
(Draper & Brown, 2002). The first case studies of CRS were predominantly from physics 
professors. Now, case studies can be found in almost every area of education from 
business to medical institutions. Additionally, CRS are being used not only by 
universities, but also in elementary and K-12 settings (Johnson & McLeod, 2005). The 


focus of this study will be on a university environment. 


Professors have one of two options when preparing questions to be used in class 
with CRS. First, the advised approach is to prepare questions to be used in the lecture 
slides during the lesson plan. Second, CRS provides teachers with the ability to create 
questions “on-the-fly” (Renaissance Learning, 2007). That is, during the lesson plan, if 
an important concept comes up that the professor wants to make sure is understood by the 
entire class, the professor can simply test the entire class by pausing and typing the 
question using the CRS software. Note that the first option is preferred, because taking 
time to prepare well-thought-out questions better ensures that students are grasping 


important concepts (Caldwell, 2007). 


Varying views are found among CRS experts on how many questions a professor 
should ask his or her class during a 50-minute class period. Typically, between two to 
five questions are given per 50 minutes of class instruction (Burnstein & Lederman, 
2001). Using this rule of thumb, professors can properly format their lectures so that they 


are not taken over by excess usage of CRS. 
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Once a professor chooses to use CRS technology, he or she should take some time 
before the semester or quarter begins to become familiar with how to set up and use the 
CRS. CRS software has a slight learning curve. New users should be sure to set aside 
some time to create class rosters and sessions with well developed questions (Hoffman & 
Goodwin, 2006). Once one masters the software, the main challenge is developing 
effective questions. Practice makes perfect and saves time. The local IT departments 
should be able to install the software on the computer that the professor would be using 
for lectures. A good idea is to devote a laptop with the software that can be transported 
from one classroom to the next (Hoffman & Goodwin, 2006). Second, software tutorials 
are available through most major CRS companies, or a resident expert should be able to 
explain the basics. Adding questions is as simple as creating Microsoft PowerPoint slides 


for a presentation or creating transparencies to be used on an overhead projector. 


On the first day of class, the professor should take some time and explain in what 
capacity clickers will be used during the semester or quarter and conduct a familiarization 
session with the class. Although these devices are fairly easy to use, professors should 
run through example questions to familiarize the class with the new technology and ease 
any “technophobia” that some students may have about an advanced technology (Berube, 


2006). 


The Question Cycle below (see Figure 1) is an excellent representation of how a 
question is sent by a professor, discussed by the students, displayed and discussed by the 
professor and, eventually, evaluated by the professor and revised, if necessary (Dufresne, 
Gerace, Leonard, Mestre, & Wenk, 1996; Wenk, Dufresne, Gerace, Leonard, & Mestre, 
1997). 
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Figure 1. | The Question Cycle—an Effective Model for CRS Use in Class [From Wenk, 
Dufresne, Gerace, Leonard, & Mestre, 1997] 


The typical question format is as follows: 

1. Professor sends a question to the class by Microsoft PowerPoint, overhead 
projector or writing the question on the screen. 

2. Students take a few minutes to decide upon an answer either individually or in 
groups. 
Students then enter their answer into their clickers. 


se) 

4. Tallied responses are displayed on a projection screen. 

5. Professor discusses the possible solutions and supporting evidence. 
6 


Correct answer is given. 


With deft management, this process can be turned into a lively interchange of 
ideas and arguments between students. Instructors can follow up with a brief lecture on 


the relevant point (Beatty, 2004). 
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CRS have been used by professors in an assortment of ways. The following list 


outlines some of the more common uses by professors: 


1. 


Attendance — Typically a non-issue at NPS, but difficult in the larger 
classrooms; if clickers were used, they would accelerate the process. The 
disadvantage is that the ID number on each clicker has to be mapped to an 
individual student. 

Assessing understanding — Before the lecture starts, CRS can be used to assess 
the level of understanding of a particular topic, identifying areas that have 
already been sufficiently covered and highlighting sections in need of 
particular attention (Simpson & Oliver, 2006). 

Initiating discussion — One of the main uses of CRS is to stimulate interest and 
initiate discussion (Simpson & Oliver, 2006). Professors question the class, 
giving ample time for students to talk among themselves before answering on 
their clickers. 

Quizzes or tests — Reports of using clickers for summative high-stake testing 
are relatively rare. Quiz questions can check whether students are: 
comprehending the material, actively thinking or paying attention (Caldwell, 
2007). Professors can also take a less direct approach and count the quiz 
grades for a small portion of the student’s grade. For example, clicker quizzes 
at NPS will count for five percent of a student’s participation grade. 

Exercise problems — Professors could give short exercise problems during 
class to test comprehension. 

Creation of community — At the beginning of a semester or quarter, CRS can 
be used to create awareness in a group or class. This can contribute to the 
forming of the group identity and support any future teamwork (Simpson & 


Oliver, 2006). 


Like any technology, when used incorrectly, CRS can do more damage than good. 


CRS require professors to come up with innovative methods in designing questions that 


stimulate active learning. As Draper, Cargill, and Cutts (2001) observe: 
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Instructional design mostly is not in the equipment or software, but in how 
each teacher uses it. That is a lesson which perhaps the rest of the learning 
technology field should take more to heart if the aim is in fact, to improve 
learning rather than to promote the glamour of machines. 


For example, creating questions that are too simplistic can give professors the 
false belief that students comprehend a complex concept. This can be troublesome if the 
professor continues to the next topic, leaving the class confused. Professors can 
incorporate a failsafe mechanism for comprehension issues through the use of clickers. 
For example, the professor can enable a button on the clickers, so that when pressed by 
students, an alert message would appear on the professor’s PC notifying the professor a 
student has a problem with comprehension. This would be similar to students raising their 
hands, but by using clickers, the student’s anonymity would be protected. If multiple 
students respond in this manner at a particular point in the lecture, the professor can 
choose to either address the comprehension issue immediately or wait until later for 


further explanation. 


E. COMPARISON AND ANALYSIS 


Computer response systems (CRS) offer a variety of functions. As with other 
factors, the educator must determine the best capacity in which CRS suits needs within 
the class by choosing the functions it requires. Amid the numerous models, this study 
compares and analyzes five CRS models that use radio frequency (RF) technology. The 
following models were selected for their popularity among major universities: Qwizdom, 
Renaissance, eInstruction, Interwrite PRS, and TurningPoint. For a comparative analysis 
of these popular models, refer to Table 1 (pages 24-25). Each of these brands was 
compared on various characteristics: response capabilities, system requirements, and 


overall functionality. 


Keypad design is one of the most critical characteristics for students using CRS. 
Students are often concerned whether their responses to questions were even transmitted 
to the receiver. Qwizdom, Renaissance, elInstruction, Interwrite PRS, and TurningPoint 
models all have LCD screens, which help students know that their answers were 
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submitted. Using TurningPoint, students can input fractions, decimals, and negative 
numbers, and they receive instant right/wrong feedback on each device (Turning 
Technologies, 2007). This immediate feedback is a vast improvement from the earlier 


models that lacked a screen and only offered basic answer response methods. 


Although earlier CRS models only had the capabilities for students to answer 
yes/no, true/false, or A/B questions, this is no longer the case. The elnstruction clicker 
allows for numeric entries of up to 12 characters that can be entered and viewed on the 
three-line LCD screen. The symbol button makes it easy to answer higher-level math and 
science questions (e/nstruction, 2007). Interwrite PRS leads the other CRS with their nine 
different types of answer choices: multiple choice, true/false, yes/no, numeric, short 
answer, multiple correct, rank order, decimal point, and fractions (Interwrite Learning, 


2007). 


Each of the systems evaluated is compatible with Windows and Macintosh 
operating systems. One CRS stands out from the rest in operating system compatibility. 
In addition to being compatible with Windows and Macintosh operating systems, the 
Interwrite PRS system is also compatible with the Linux operating system (/nterwrite 
Learning, 2007). Although Linux is not as popular as Windows or Macintosh, 
universities using Linux will not have to spend much time narrowing their choices on 
selecting their CRS of choice. These fortunate few universities only have one option, the 


Interwrite PRS. 


In addition to working with Microsoft Office PowerPoint, some of the companies 
have developed software packages to better adapt their product to subject-specific 
courses. Qwizdom, for example, has a program called Qwizdom Actionpoint, which is an 
easy-to-use toolbar that fits comfortably within Keynote or Microsoft Office PowerPoint. 
This allows the user to instantly create an interactive slide presentation, view response 
graphs, play media files, and control the entire presentation with the Qwizdom presenter 
remote (Qwizdom, 2007). TurningPoint created a bundle called TurningPoint AnyWhere 
software. AnyWhere polling application uses the same functionality as TurningPoint 


through a floating toolbar that allows users to poll from content in any PC application, 
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including web browsers, PDFs, Word documents, Blackboard, and more (Turning 
Technologies, 2007). One drawback of the separate program is the screen requirement the 
“floating” window takes up that often blocks a portion of the presentation material 


(Barber & Njus, 2007). 


The implementation of classroom response systems allows a lecturer to record the 
data collected from each student to evaluate the student’s progress. Each CRS provides a 
different report system, but all record answers (Barber & Njus, 2007). In addition to 
being a CRS with several response methods, the Renaissance doubles as a calculator and 
can store information (homework assignments and notes). The large LCD screen with 
seven lines of text allows students to view entire questions at one glance without having 
to scroll down small screens with only one or two lines of text, which is a common 


characteristic among other CRS (Renaissance Learning, 2007). 


Cost is a function that is very difficult to measure with these systems, because the 
models examined vary drastically in both functions and software availability. A lower 
cost Interwrite PRS clicker costs around $30 for orders of 100, whereas the Qwizdom Q4 
Handheld is upward of $122 per unit (Renaissance Learning, 2007). Discounts can 
typically be obtained by buying clickers in bulk. Pricing per CRS depends on 


requirements and needs of the buyer’s organization. 


When purchasing the CRS of choice, manufacturers rely heavily on customizing 
their CRS for individual users to establish a relationship for further business. 
Manufacturers will send representatives or sales consultants to greet the buyer and sit 
down and discuss the buyer’s requirements. Typically, this should be a person from the 
buyer’s organization who is knowledgeable about organizational IT specifications and 
limitations, as well as the educational needs and requirements of the organization. Upon 
reaching an agreement about which CRS will be purchased, including how many clickers, 
receivers and associated software packages are required, a cost can be determined; at that 
point, the sales consultant can offer a price. Therefore, comparing total costs is a very 
difficult process, because actual costs are dependent on the needs and requirements of the 


buyer’s organization. 
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Due to NPS’s unique education system with various types of classrooms and 
material being taught, the CRS must be dynamic to meet the professors’ needs. For this 
reason, the CRS with the most diverse software package is recommended. TurningPoint 


CRS is the best choice, based on the results from this comparison and analysis. 


TurningPoint CRS has many strong features. First, TurningPoint integrates fully 
into the Microsoft Office Suite, including PowerPoint, Word, Excel and even Outlook. 
TurningPoint’s integration with Microsoft Office PowerPoint gives trainers and 
presenters the ability to author, deliver, assess and report without having to leave 
PowerPoint (Turning Technologies, 2007). Secondly, TurningPoint CRS easily allows 
professors the ability to monitor how their classes are responding to questions to 
determine if more time is needed on a specific topic. Finally, finding a CRS that 
seamlessly integrates into existing NPS technology architecture such as Blackboard, 
Microsoft Office Suite and the more popular educational publishers such as Thomson 
Learning and Glencoe/McGraw Hill is extremely important (Turning Technologies, 
2007). Although their LCD screens are large, a major disadvantage of TurningPoint CRS 


is the limited two lines of text. 


Despite different schools of learning and faculty members preferring different 
models, it is important that NPS select a single CRS manufacturer. NPS needs to learn 
from the mistakes of other universities when purchasing multiple CRS and being forced 
to standardize to one manufacture at some later time. Creating a committee with 
representatives from each of the major schools of learning is the recommended approach 
used by other universities such as Ohio State University. Ohio State University put 
together a CRS committee to perform the following actions: 

1. Define the key educational goals. 

2. Propose technical solutions and evaluate costs. 

3. Discuss policies, procedures and guidelines for “clicker” deployment (Metros, 


2005). 


By using a single CRS manufacturer such as TurningPoint, faculty and students 


can adapt faster to using the system, technical support requirements can become uniform, 


pie 


creating a strong in-house support system (Barber & Njus, 2007), and costs can be 
reduced. NPS students can be trained to use one standardized CRS during their 
indoctrination week. This would enable students to develop their knowledge base of 


clickers to enhance their educational experience. 
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Table 1. 


A Comparison of Different Classroom Response Systems [From Bhuta, 2006] 











Manufacturers 
Features Interwrite PRS eInstruction Qwizdom TurningPoint Renaissance 
Type Radio frequency Radio frequency Radio frequency Radio frequency Radio frequency 





Computer System 


Windows or Mac or 


Windows or Mac 


Windows or Mac 


Windows or Mac 


Windows or Mac 
































Compatibility Linux 

LCD Screen Yes. Two lines of text. | Yes. Three lines of text | Yes. One line of text. Yes. Two lines of text. Yes. Seven lines of 
text. 

Communication 150 ft. 200 ft. 1000 ft. 200 ft. 150 ft. 

Range 

Units per receiver Up to 2047 Up to 1000 Up to 1024 Up to 1000 300 

Battery 3 AAA batteries 2 AA batteries 2 AA batteries 2 coin cell batteries 2 AA batteries 

Battery Life Up to 20 weeks 360 hours Up to 12 months Up to 12 months Up to 12 months 

Instructor control Keyboard/mouse Keyboard/mouse Instructor remote Keyboard/mouse Keyboard/mouse 

Presentation Software | Limited PowerPoint PowerPoint PowerPoint PowerPoint, Turning PowerPoint/AccelTest 

Integration Point 

Student device Blinking light Blinking light Alphanumeric display Blinking light Blinking light 

displays feedback 

Supports paper-based | Yes No Yes No No 


testing 




















2 




















Table 1. (continued) 
Manufacturers 
Features Interwrite PRS eInstruction Qwizdom TurningPoint Renaissance 
Question types 9 types: Multiple 7 types: Multiple 7 types: Multiple 6 types: Multiple 6 types: Multiple 
available choice, T/F, Yes/No, Choice, T/F, Yes/No, Choice, T/F, Yes/No, Choice, T/F, Yes/No, Choice, T/F, Yes/No, 
numeric, short answer, Rating Scale, Numeric Numeric, multiple- Rating Scale, Numeric, Numeric, Decimal 
multiple correct, rank Decimal/fraction mark multiple choice, Decimal 
order, decimal point, rating scale, sequencing 
fractions 
Publishers using (This | Pearson Group Allyn & | Pearson Group Pearson Group Wiley Pearson Group Thomson | McGraw Hill, 
list keeps on changing) | Bacon Thomson Wiley, | McGraw-Hill Pearson Prentice Hall School, 
Worth W. H. Freeman Glencoe 
Gradebook feature Yes Yes Yes Yes Yes 





Student cost/unit 


$30 (orders of 100+) 
No access code 


$3 Access code: 
$18/qtr 


$50-$122, dep. on 
volume. No access 
code 


$50, dep. on volume. No 
access code 


$79.99. No access 
code 





University cost 


Varies, depending on Publisher agreement 








Website 





http://www. interwritele 
arning.com 





www.einstruction.com 





www.quizdom.com 





turningtechnologies.com 





www.renlearn.com 
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I. SYSTEM REQUIREMENTS AND RENAISSANCE CRS 
SOFTWARE FUNCTIONALITY 


A. SYSTEM REQUIREMENTS 


NPS purchased the Renaissance Classroom Response System, as it contained the 
hardware and software that best met NPS needs. Inside each Renaissance Classroom 
Response System carrying case is the following software packages and instructions: 
AccelTest version 1.0 (CD), 2KnowToolbar version 1.0 (CD), and online training 
instructions. Additionally, every system includes a wireless receiver, USB cable to 
connect the wireless receiver to a PC, and 24 Renaissance Responder handheld remote 
units or clickers. Finally, each case includes user manuals and the necessary access codes 


to register the equipment and download updates, patches, and utilities. 


The Renaissance Classroom Response System operates on Windows or 


Macintosh. The system requirements for these two operating systems are as follows: 


Table 2. Renaissance CRS System Requirements for Windows and Macintosh [From 
Renaissance Learning, 2007] 





Windows Macintosh 





Processor: Pentium P3 450 MHz or faster Processor: Power PC G3 450 MHz or faster 





Operating Systems: Windows 2000 or later | Operating Systems: System 10.3.9 or later 

















RAM: 128 MB physical RAM RAM: 128 MB physical RAM 
Hard Drive Space: 40 MB Hard Drive Space: 40 MB 
CD-ROM: Drive Required CD-ROM: Drive Required 
Pointing Device (mouse): Required Pointing Device (mouse): Required 








Renaissance Classroom Response System software must be installed on the PC to 
which the receiver is connected for the system to operate properly. Installation requires 


the receiver driver, which is included when installing AccelTest or 2KnowToolbar. All 
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the classrooms at NPS in the Graduate School of Business and Public Policy (GSBPP) 
are configured with PCs at the front of the classroom that meet the necessary system 
requirements to run Renaissance. This includes the thin client server. Due to NPS’s IT 
restrictions, contacting the IT department to load the Renaissance CRS software on 
school computers is required. Unfortunately, the CD for AccelTest version 1.0 is 
outdated. An updated version of AccelTest, presently version 3.0, can be downloaded 
from the Renaissance website, http://download.renlearn.com/us/search.asp?type=2. 
NPS’s access code is required for the download and can be obtained from the GSBPP 


Director of Instructional Technology. 


The Renaissance website, http://www.renlearn.com/Profdevel/PD/ 
2Know_pd.aspx? type=ondemand&product=2know, provides an excellent step-by-step 
visual online tutorial on how to install the software. Additionally, the website provides 
specific step-by-step instructions and training on how to use AccelTest and 
2KnowToolbar (Renaissance Learning, 2007). The website can also be used for those 
professors who choose to install the AccelTest or 2KnowToolbar (or both) on their 
personally owned PCs. After loading AccelTest on any PC, a PDF version of the user 


manual can be opened from the start menu. 
B. RENAISSANCE CRS SOFTWARE FUNCTIONALITY 


The primary difference between the 2KnowToolbar and AccelTest is the 
interface. The 2KnowToolbar is used for quick polling of the audience while AccelTest is 
a more formal assessment. The 2KnowToolbar is a floating toolbar that remains on top of 
other open software applications on the desktop of a PC. Three different types of 
questions can be asked using the 2KnowToolbar: true/false (T/F), numeric (123), or 
multiple choice (ABC). Student responses and answers can easily be displayed in a bar 
graph format on a projector screen for the class to view. Additionally, the toolbar 


provides a feature called “pulse,” which allows professors to poll the class during a 
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lecture, or students to use the yes/no buttons to register if they comprehend the material. 
Professors can use the feedback provided to either move on to the next topic or address 


the concept from another direction. 


The advantages of 2KnowToolbar are that it is easy to install and requires little 
time to learn. With this software, clickers do not have to be assigned to individual 
students. The main disadvantage is that this software application does not record student 
responses. Recording student responses may be important if professors wish to review 


student responses at a later time to modify their lectures or for grading purposes. 


AccelTest, a more advanced software application, provides the extra features that 
the 2KnowToolbar lacks. AccelTest allows professors to set up individual classes and 
create sessions to assign homework assignments, give quizzes, and even give formal 
exams. The grade book feature allows professors to keep track of individual student 
scores. The report feature allows professors to view and print a summary of students’ 
grades, either individually or collectively as a class. Although AccelTest would be 
installed on lecture podium PCs in Ingersoll, transferring the data from the classroom PC 
to a personally owned PC can become tedious and time consuming. An alternative is for 
professors to install and use AccelTest from their personally owned PCs. This will 
require professors to connect their personally owned PCs through a VGA connector to 


the classroom overhead projector for each class they wish to use AccelTest. 


The advantages of AccelTest is that it helps professors create questions and 
answer choices that can be viewed on students’ clickers, which include an easy scroll 
feature that accommodates longer questions. This eliminates the need for paper, 
projector, or TV. Using AccelTest, professors can create a nearly endless supply of 
reusable quizzes, tests, and other exercises to assess students’ performance in virtually 
any subject. With AccelTest, professors save time by automating assessment scoring with 
results recorded immediately to the grade book. As a result, professors spend less time on 


paperwork and record-keeping tasks (Renaissance Learning, 2007). 


AccelTest does not come without its disadvantages. First, the software application 


takes time to become familiar with the features and functionality, which may discourage 
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its use by busy professors. All assignments, quizzes, and tests have to be inputted into the 
software application, which can also be time consuming. Second, clickers have to be 
assigned to individual students. Therefore, professors must let the students keep the 
clickers in their possession or clickers must be collected and redistributed to their 
assigned owner for each class. When professors utilize the homework assignment feature 
students must retain the clickers. This feature enables students to take the clickers home 
and complete homework assignments on their own time. When students bring their 
clickers back to class, the clickers are synchronized through the receiver to AccelTest and 
homework assignments are downloaded from the clickers to AccelTest and automatically 


graded. Results are viewed in the grade book tab or in the reports drop-down menu. 


30 


Hl. METHODOLOGY 


A. PARTICIPANTS 


Participants were 41 students enrolled in a graduate information technology (IT) 
course at the Naval Postgraduate School. The course is designed to make students more 
effective IT users and decision makers, and help them recognize opportunities where the 
application of IT solutions can provide a strategic advantage. These objectives are met by 
providing students with a broad overview of computer technology, information systems, 
database/knowledge management, networks, and information security. As a first-quarter 
course, it was the first traditional classroom course in several years for many of the 


enrolled students. 


Participants ranged in age from 27 to 50 years old (M = 33.02, SD = 5.36), and 37 
were male (90.2%). Participants included two civilians (4.9%), 13 international military 
officers (31.7%), and U.S. military officers (63.4%) with the following ranks: 12.2% 
were O2, 51.2% were O3, 24.4% were O4, 4.9% were O5, and 2.4% were O6. 
Participants identified themselves as African-American (4.9%), Asian (7.3%), Caucasian 
(56.1%), Hispanic (12.2%), or “other” (7.3%), and 7.3% declined to report their 


race/ethnicity. 
B. DESIGN 


The current study employed a posttest-only independent groups quasi- 
experimental design to test the effects of clickers in the classroom. Specifically, two 
sections of the IT course were included in the study, and the researchers chose one 
section randomly to serve as the experimental classroom, and the other to serve as the 


control classroom. 


Each student in the experimental classroom was given a Renaissance Responder 
(clicker) at the beginning of each lecture period and asked to log into the 2Know! Session 


(clickers were not assigned to specific students). To avoid the novelty of the response 
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system impacting the outcomes of interest, the clickers were used in the experimental 
classroom for two weeks prior to data collection to resolve technical difficulties and 
condition students to the technology. During this period, the students were trained how to 
use the clicker and were able to ask questions until each student felt comfortable with the 
technology. The students in the control group were not issued clickers or introduced to 


the technology. 


C. PROCEDURE 


Throughout the ten-week quarter, student interactions were observed in both 
classrooms. The total number of student interactions that occurred each hour during nine 
randomly selected two-hour class sessions resulted in an hourly interaction score. 
Individual student scores were not recorded. An interaction occurred when a student 
participated in one of the following ways: raising of hand, answering a question (from the 
professor or another student), asking for further explanation, or responding with the pulse 
function of the clicker (test group only). Only voluntary interactions were recorded and 
all “polling” questions to the class were excluded from the interaction score (because this 


interaction was instructed by the professor). 


When the clicker’s pulse function was used in the experimental classroom, 
students would indicate if they understood the concept(s) being presented by the 
instructor (a positive response) or did not understand (a negative response). Students 
were instructed that they could respond as many times as they liked (e.g., they could 
change from a negative response to positive response if the issue was resolved). If the 
pulse data indicated that the concept was still unclear (e.g., most responses were 
negative), the instructor asked students to explain what questions existed (if a student 
responded to this question, it counted as an interaction). After a course concept was 
presented in full and the instructor observed a large majority of responses, the results 
from each “pulse window” were evaluated. After any issues were resolved, the pulse 
function was reset to zero (1.e., all pulse data would be erased and students could begin 


the process again on the next course topic). 
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In addition to tracking interaction scores, students in both classes completed a 
survey. After providing informed consent, participants completed items to measure 
student motivation, perceived teacher immediacy, course liking, and levels of student 
engagement. Additionally, the experimental group completed items to assess their beliefs 
about clickers, and answered open-ended items where participants provided comments 
and suggestions regarding the clickers. Finally, both groups completed demographic 


items. 
D. INSTRUMENTATION 


Measures were scored such that higher scores indicated greater perceptions of the 
construct being measured (see Appendix for measures). Given that specific items were 
specified a priority to measure only one factor, confirmatory factor analysis was 
employed to test the measurement model (Anderson, Hunter, & Gerbing, 1987; Hunter & 
Gerbing, 1982; Levine, 2005). The data was found to be consistent with the proposed 
factors. Internal consistency tests showed that the errors calculated between items 
measuring the same construct were within sampling error of zero. Likewise, the 
parallelism test indicated that the errors calculated between items measuring different 


constructs also were within sampling error of zero. 


Course liking. Five 7-point Likert-type items taken from Jackson and Trees 
(2003) measured students’ liking of the course. Student-reported course liking had a 
mean of 6.04, (SD = 0.84, a = .95). 


Overall evaluation of clickers. Respondents in the test group completed 12 7- 
point Likert-type items modified from Fitch (2004) and Greer and Heaney (2004) 
designed to measure student evaluations of clickers. This scale included items such as 
“Using the clicker added interest to the class” and “The clickers increase interaction in 


the classroom.” Overall clicker evaluations had a mean of 5.13, (SD = 1.42, a = .97). 
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Perceptions of clicker usefulness. Participants in the experimental group 
completed four 7-point Likert-type items designed to assess the degree to which students 
perceived the clicker’s functions to be useful. Perceptions of clicker usefulness had a 


mean of 5.44, (SD = 1.49, a = .98). 


Perceptions of function liking. Participants in the experimental group completed 
four 7-point Likert-type items designed to assess the degree to which students liked the 


clicker’s functions. Liking of clicker functions had a mean of 5.38, (SD = 1.51, a =.97). 


Interaction score. The total number of responses (i.e., hands raised, clicker 
response, verbal response) during each class hour were summed to create the interaction 
score. Student interaction scores ranged from 0-17 per hour and had a mean of 8.17 (SD = 


5.51). 


Student motivation. Twelve 7-point semantic differential items taken from 
Christophel (1990) were used to measure student motivation. Student motivation had a 


mean of 4.92 (SD = 0.74, a = .91). 


Student engagement. Four 7-point Likert-type items modified from Jackson and 
Trees (2003) measured the amount of student engagement. Students’ self-reported 


engagement had a mean of 5.34, (SD = 1.08, a = .87). 


Perceived teacher immediacy. Four 7-point semantic differential items taken from 
Kearney, Plax, Smith, and Sorenson (1988) measured student perceptions of teacher 


immediacy. Perceptions of teacher immediacy had a mean of 6.44 (SD = 0.78, a = .93). 


Open-ended items. Respondents in the test group were asked to respond to two 
open-ended items designed to elicit general feedback about their clicker use. Specifically, 
participants received the following instructions: 

Now we would like you to take a moment and provide any additional 


feedback you’d like to share about the clickers. Please feel free to 
comment on any aspect—likes, dislikes, any comments for future use. 
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IV. RESULTS 


A. COURSE LIKING 


Results indicated that the degree to which students using clickers liked the course 
(M = 6.04, SD = .84) differed significantly from students in the control group (M = 5.34, 
SD = 1.36), t(39) = 2.04, p < .05, r=31. 


B. EVALUATION OF CLICKERS 


Because only students in the experimental condition were able to evaluate the 
clicker and its functions, a one-sample f-test was used to determine if students’ 
evaluations were more favorable than the midpoint of the scale. Results indicated that 
students’ overall evaluation of the clickers (M = 5.13, SD = 1.42) were substantially more 
favorable than the midpoint of the scale (4.00), as indicated by the large magnitude of the 
effect size, (23) = 3.89, p< .01, r= .63. 


C. EVALUATION OF CLICKER USEFULNESS 


One-sample f-test results indicated that students rated the usefulness of the 
clicker’s functions (M= 5.44, SD = 1.49) as substantially more favorable than the 
midpoint of the scale (4.00), as evidenced by the robust effect size, (23) = 4.72, p < .001, 
r=.70. 


D. EVALUATION OF FUNCTION LIKING 


One-sample f-test results indicated that students rated their liking of the clicker’s 
functions (M= 5.38, SD = 1.51) as substantially greater than the midpoint of the scale 
(4.00), as indicated by the large effect size, 1(22) = 4.38, p < .00O1, r= .68. 
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E. INTERACTION SCORE 


Results indicated that the total interaction scores for participants in the 
experimental group (M = 9.33, SD = 7.16) did not differ substantially from those in the 
control group (M = 7.00, SD = 3.20), (16) = 0.89 p = .39, r =.22. 


Results indicated that the number of hands raised each hour in the experimental 
group (M= 4.33, SD = 2.35) did not differ significantly from the number of raised hands 
in the control group (M = 7.00, SD = 3.20), 7(16) = -2.02, p =.06, r=-.45. 


F. STUDENT MOTIVATION 


Independent sample f-test results indicated that student motivation for students 
using clickers (M = 4.92, SD= .74) did not differ significantly from students in the 
control group (M = 4.68, SD = 1.46), (36) = .67, p= .51,r=.11. 


G. STUDENT ENGAGEMENT 


Independent sample f-test results indicated that student engagement for students 
using clickers (VM = 5.34, SD = 1.1) did not differ significantly from students in the 
control group (M= 5.0, SD = 1.4), (39) = .97, p= .34, r=.15. 


H. PERCEIVED TEACHER IMMEDIACY 


Independent sample f-test results indicated that perceived teacher immediacy for 
students using clickers (M = 6.43, SD = .77) did not differ substantially from students in 
the control group (M = 6.37, SD = .82), t(39) = .28, p = .78, r= .04. 


I. OPEN-ENDED ITEMS 


The experimental group responded favorably to the clickers through the open- 
ended survey questions. Student feedback indicated that the clicker helped in learning the 
material, focusing the class, and gauging individual levels of understanding. The 
feedback also indicated that the clicker was viewed as both a useful teaching and learning 


tool and should be implemented in additional courses. Students identified some clicker 
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limitations and made recommendations for further implementation at NPS. Among these 
recommendations: using them in “a more concerted effort...would improve its 
effectiveness” and “the professor needs to be more interactive with [the] clicker.” In 
addition to the usefulness of the clickers, students described the clickers as a “fun 


learning tool and interactive experience.” 


In addition to the clickers increasing course liking, the students found the clicker 
to be useful. One reoccurring theme from the open-ended survey responses was that the 
usefulness of clickers decreased when the technology was not working properly due to 
user errors. Students suggested that the instructor should understand the clicker system 
fully before using it in class, because it was distracting when time was taken to 


troubleshoot the system. 
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Vv. DISCUSSION 


The evaluation of clickers in the classroom was done primarily to determine if 
interaction was increased. Interaction is crucial in classrooms, because increased 
interaction is linked to increased learning. Past studies report that students learn in a 
myriad of ways and it is up to the instructor to promote as many means to transport the 
material to the students as possible. This study examined how clickers affect students 


with the following research questions: 
Q1: Do clickers increase course liking? 
Q2: How do students evaluate the clickers? 
Q3: Do students find clickers useful? 
Q4: What functions do students like? 
Q5: Do clickers increase classroom interaction? 
Q6: Do clickers increase student motivation? 
Q7: Do clickers increase student engagement in the course? 
Q8: Do clickers increase perceived teacher immediacy? 


The results of the study conclude that the implementation of clickers in a 
classroom increased course liking. Students found the clicker functions to be both useful 
and favorable. In addition, although not proven statistically, data values indicated positive 
correlation between clickers and increased participation. The study failed to conclude that 
clickers increase student motivation, perceived teacher immediacy, and student 


engagement. 
A. COURSE LIKING 


Results indicated that the degree to which students using clickers liked the course 
differed significantly from students in the control group. Not unlike past studies, the 
implementation of the clickers proved to increase the liking of the course (Abrahamson, 
1999). 
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Some of the reasons for this increase include the greater amounts of interaction 
and feedback. Through addition of the clickers, students had means to communicate to 
the professor their levels of understanding, so that the rate of material could be adjusted 
accordingly. Additionally, the clickers facilitated another method for students to 


communicate to the instructors and classmates. 


The increased communication in the experimental group may have given the 
participants an increased feeling of learning. Knowledge is constructed in response to 
interactions with others (Brown, 1989). Through interactions with classmates and 
professor, and having immediate feedback given through the clickers, individuals were 
given many methods to actively learn, increasing their knowledge base. Despite the 
implementation of clickers, which could reduce verbal communication, the instructor 
used the responses as a launching point for further discussion. After students responded, 
the instructor asked if someone would want to explain how they came to the correct 
answer. To ensure the majority of the class was on the same level, a second question was 


posed to ensure everyone had better understanding of the material. 
B. EVALUATION OF CLICKERS, USEFULNESS, AND FUNCTIONS 


Students reported that all functions of the clicker were useful. Through data 
obtained from experiment participants, it was concluded that the numeric response was 
the top function (in both usefulness and liking). This response was somewhat unexpected; 
however, this feature added significant value in answering IT-related questions. Rather 
than the students getting two minutes to work through a problem and then revealing their 
answer to the entire class one by one (with the ability to change their answer if others are 
not reporting it), all students were able to work through at the same time and report their 
answer. The instructor kept the inputs hidden until all had answered or time was called. 
Not only did this save time in the class, it also allowed students to work through the 


problem without being influenced by other answers. 


Despite the statistical report stating that the clickers were useful and beneficial, 


some participants were adamantly against the technology. According to one survey the 
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“clickers were not used in any type of profound way that advanced the educational 
process.” Some students also felt that their infrequent use slowed the progress of the 
class. Although the technological glitches affected the entire class, students were free to 
do as they wanted. The technology allows those who benefit from the clicker to 
participate, and those who do not to decline. The students in the latter group can rely on 


the traditional methods of interacting within the class if they so prefer. 


1. Interaction Score 


The difference between the experiment and control group interaction scores was 
not statistically significant. However, the means were in the direction of hands raised in 
the control group, and although the significance test did not meet the necessary threshold, 
the effect size is substantial; it is likely not significant simply due to the meager sample 
size. The researchers of this study hypothesize that with additional testing, with a larger 
sample size, the difference would be significant. The experimental group interacted more 
with the clickers (M = 9.33) versus hands alone (M = 4.33). This increase could be for 
several reasons. First, with a clicker the student has an additional way to interact. Second, 
when using the clickers the students are given immediate feedback through viewing of 
everyone’s response. This encourages participation when a question is posed to check 
their comprehension level. Third, when a question does arise, students can anonymously 
indicate through the clicker that something is unclear. Given the competitive nature of the 
students attending NPS, the clicker fosters healthy classroom competition by involving 
everyone in the process, even those who may not normally participate, without the fear of 


being singled out or answering incorrectly. 
C. STUDENT MOTIVATION AND ENGAGEMENT 


There was no significant difference between the experimental and control groups, 
indicating that the clickers did not affect student motivation or engagement. Despite the 
clickers being an additional method for the instructor to stimulate the students, no 
significant motivational differences were recorded. This does not, however, indicate that 
students were not learning (Christophel, 1990). The study took place in a Fundamentals 
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of IT course. The format of the class consisted primarily of lectures, periodically broken 
up by questions allowing students to listen and take notes on the material. Students in the 
course typically took notes and memorized the concepts prior to the exams. Although 
they learned the material, they did not have to be highly motivated throughout the term, 


just motivated enough to learn the material for the exams. 


Data did not produce a statistically significant difference between the groups, 
possibly attributing to the type of students participating in the study. Past studies 
primarily implementing the technology in undergraduate courses indicate that clickers 
increase motivation and engagement (Trees, 2007). Students at NPS are highly motivated 
individuals. The students are at NPS due to their military successes and academic 
achievements. Due to the nature of the school, most of the students realize that failure to 
succeed will have negative implications on their career. Given the nature of the 
participants and the incentives to do well, and the high scores reported for these survey 


questions, a ceiling effect is possible. 


Student engagement is also not very difficult at NPS, as most class sizes are under 
30. Despite this small class size, there is still only one instructor who must encourage 
participation from each of the students. Again, due to the nature of the study participants, 
many do not have problems engaging in the class discussion and actively participating. 
At NPS. However, there is a different aspect of participation that is not commonly seen in 
traditional universities: rank. As a military institution, officers are sent at various times of 
their career. Typically officers attend NPS at the O-3 level; however, in this study O-3 
ranking officers made up approximately 51% of the participants. The other participants 
included O-4 (24%), O-2 (12%), and a small number of O-5, O-6, and civilians. One of 
the goals at NPS is to prevent rank from impacting learning in the classroom; however, 
since the hierarchical structure is ingrained heavily in each of the officers, it is often 
difficult to separate from it when in the classroom. On occasion, during course discussion 
lower ranking officers will cease to speak their opinion when a higher ranking officer 
voices another opinion. The clickers are able to reduce this aspect further by allowing 


opinion polls to be taken anonymously. 
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D. PERCEIVED TEACHER IMMEDIACY 


Results indicated that perceived teacher immediacy did not differ from students in 
the control group. Due to the similarity of scores from each group, the clickers did not 
affect the perceived teacher immediacy. The scores given by both groups were extremely 
high. These results indicate that the instructor exhibited verbal and nonverbal immediacy 
behaviors. High levels were reported, even with the experimental group where it was 
hypothesized that nonverbal immediacy would be lost. These results suggest that, in this 
test, the professor uses more verbal immediacy than nonverbal. Classroom observation of 
the class revealed that humor and personal stories were weaved into the lectures. The 
instructor also knew the majority of students by name and encouraged interaction 


throughout the class. 


The instructor’s high immediacy score not only enabled students to communicate 
further with the instructor, but also affects the learning outcomes (Witt, 2004). Although 
how immediacy increases learning has not yet been agreed upon, past studies have 
proposed several explanations. Christophel (1990) proposed that immediacy increases the 
student’s motivation to learn, which increases learning. Immediacy may attract student 


attention, which is related to cognitive learning (Comstock, 1995). 


E. LIMITATIONS OF THE STUDY 


Through the course of the study, it was determined that further testing is required 
to determine the how to achieve maximum benefit of the clickers. Additional tests 
suggested by the researchers are: different types of course material, larger sample sizes, 
more consistent use, and even use of all function capabilities. Another factor that should 
be better controlled are the technical difficulties, both technology and user, which distract 


from the course. 


F. RECOMMENDATION 


As seen through this study, clickers will not immediately increase student 


engagement. In fact, how the clickers impact the course depends directly on how the 
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students respond to and use them (Trees, 2007). The implementation of clickers can 
potentially reform the classroom, by increasing course liking and interaction. The attitude 
toward the clicker is an important factor when considering the introduction of the 
technology. Without student buy-in, clickers will not be used for participation and 


engagement levels may not be impacted. 


Implementation of clickers allowed students to respond anonymously, perhaps 
increasing their liking of the course. Students are able to learn by working through and 
answering questions whereas, in traditional classrooms, they would not in fear of an 
incorrect answer. The clicker also allows for each student to answer the same question 
simultaneously, while keeping the results hidden until the instructor is ready to discuss 
the correct answer. This allows students to answer at their own pace without the 
temptation of giving the same answer as faster classmates. The quick reporting ability 
allows the instructors to know immediately the understanding levels in the course, rather 


than waiting for each person to report while going around the room. 


44 


APPENDIX 


Course Liking 

Read each statement and rate how much you agree or disagree with each of the following 
statements by circling a number. | means “very strongly disagree,” 4 means “neutral,” 
and 7 means “very strongly agree.” 
1. I like this course. 

Very strongly disagree 1 2 3 4 5 6 7 _ Very strongly agree 
2.1 am happy with this course. 

Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 
3. I think this course is good. 

Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 
4. I think this course is fun. 

Very strongly disagree 1 2 3 4 5 6 7 _ Very strongly agree 
5. I am pleased with this course. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 
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Evaluation of Clickers 


Read each statement and rate how much you agree or disagree with each of the following 
statements by circling a number. | means “very strongly disagree,” 4 means “neutral,” 
and 7 means “very strongly agree.” 


10. 


I enjoyed using the clicker in class. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


Using the clicker added interest to the class. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


. The use of the clicker helped me to learn class material better. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


The clickers helped focus the class as a whole on the subject. 


Very strongly disagree 1 2 3 4 5 6 7 _ Very strongly agree 


I believe the clickers should continue to be used in this class. 


Very strongly disagree 1 2 3 4 <3 6 7 Very strongly agree 


I would like for other classes I have to use the clickers. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


. The use of clickers helped me to gauge my level of understanding of course material. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


. The clickers are fun to use. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


The clickers increase interaction in the classroom. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


Using the clickers is an effective teaching tool. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 
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11. Using the clickers is an effective learning tool. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


12. Using the clickers increased my willingness to ask questions in class. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 
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Evaluation of Clicker Functions 
Read each statement about clicker functions and rate how much you agree or disagree 


with each of the following statements by circling a number. | means “very strongly 
disagree,” 4 means “neutral,” and 7 means “very strongly agree.” 


1. The pulse reading function of the clicker is useful. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


2. I like the pulse reading function of the clicker. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


3. The multiple choice function of the clicker is useful. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


4. I like the multiple choice function of the clicker. 


Very strongly disagree 1 2 3 4 5 6 7 _ Very strongly agree 


5. The true-false function of the clicker is useful. 


Very strongly disagree 1 2 3 Le 6 7 Very strongly agree 


6. I like the true-false function of the clicker. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


7. The numeric-response function of the clicker is useful. 


Very strongly disagree 1 2 3 Ais <3 6 7 Very strongly agree 


8. I like the numeric-response function of the clicker. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 
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Student Motivation 


Please circle the number that corresponds to the word that best describes how you feel in 
this class: 


1. Motivated 1 pl 3 4 > 6 a Unmotivated 
2. Interested lf 2 3 4 5 6 7 Uninterested 
3. Involved 1 2 3 4 5 6 7 Uninvolved 
4. Not stimulated 1 2 5 4 =) 6 y, Stimulated 
5. Don’t want to study 1 2 5 4 5 6 | Want to study 
6. Inspired 1 Z 3 4 5 6 yi Uninspired 
7. Unchallenged 1 2 3 4 5 6 7 Challenged 
8. Uninvigorated 1 2 3 4 5 6 7 Invigorated 
9. Unenthused if 2 | 4 5 6 7 Enthused 

10. Excited 1 ps 3 4 5 6 ri Not excited 
11. Aroused i 2 3 4 5 6 ri Not aroused 
12. Not fascinated 1 Z 3 4 5 6 7 Fascinated 
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Student Engagement 


Read each statement and rate how much you agree or disagree with each of the following 
statements by circling a number. | means “very strongly disagree,” 4 means “neutral,” 
and 7 means “very strongly agree.” 


1. Lactively participate in class. 
Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


2. In this class, am engaged in the classroom process. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


3. Loften feel withdrawn during interactions in this class. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 
4. In this class, the students discussed course material more seriously than in my other 
classes. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 


5. Iam actively involved in this class. 


Very strongly disagree 1 2 3 4 5 6 7 Very strongly agree 
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Perceived Teacher Immediacy 


Please circle the number that corresponds to the word that best describes the teaching 
style of your teacher: 


1. Distant 1 2 3 4 5 6 7 Close 


2. Cold 1 2 3 4 =) 6 7 Warm 

3. Unfriendly 1 2 3 4 5 6 7 Friendly 
4. Withdrawn 1 2 a 4 ) 6 7 Outgoing 
5. Restrained 1 2 3 4 5 6 7 Open 
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